Virtual Reality System Offers a Wide Perspective 


Originating Technology/NASA Contribution 

R obot Systems Technology Branch engineers 
at Johnson Space Center created the remotely 
controlled Robonaut for use as an additional 
“set of hands” in extravehicular activities (EVAs) and 
to allow exploration of environments that would be too 
dangerous or difficult for humans. One of the problems 
Robonaut developers encountered was that the robot’s 
interface offered an extremely limited field of vision. 
Johnson robotics engineer, Darby Magruder, explained 
that the 40-degree field-of-view (FOV) in initial robotic 
prototypes provided very narrow tunnel vision, which 


posed difficulties for Robonaut operators trying to see the 
robot’s surroundings. Because of the narrow FOV, NASA 
decided to reach out to the private sector for assistance. 
In addition to a wider FOV, NASA also desired higher 
resolution in a head-mounted display (HMD) with the 
added ability to capture and display video. 

Partnership 

Founded by former Johns Hopkins University optics 
researcher, Dr. Lawrence Brown, Sensics Inc., received 
a NASA Small Business Innovation Research (SBIR) 
program award in 2003, and began Phase I of their FOV 
interface for NASA’s Robonaut, which also included 


development of a wide FOV camera. Baltimore-based 
Sensics created an FIMD with a wide FOV, using a 
patented optical design to combine several tiled images 
into a high-resolution, three-dimensional panorama. 
A follow-on Phase II SBIR project culminated with 
Sensics delivering the piSight HMD to Johnson Space 
Center in February 2008, after delivering the prototype 
the year before. 

The Sensics piSight HMD, essentially a high-tech view- 
finder, offers an additional panoramic, high-resolution 
camera array that allows remote control and telepresence, 
immersing the HMD operator in the robot’s workspace 
in real time. The operator sees through Robonaut’s “eyes” 
and controls its movements, working safely from a dis- 
tance. “Astronauts wearing a piSight will feel just like they 
are experiencing the world from Robonaut’s perspective, 
due to the full field-of-view, which is an exciting advance 
for the NASA robotics program,” says Brown. 

Product Outcome 

With the piSight, virtual surroundings appear in the 
viewfinder and respond to head movements. By coor- 
dinating head and joystick movements, the user flies or 
coasts up, down, forward, or sideways. Some versions of 
the piSight work with motion trackers, which eliminate 
the need for a joystick and, instead, follow the user’s 
movement realistically. 

When used with the piSight, InterSense Inc.’s 
SoniWing tracks user movements, employing technology 
similar to some video gaming systems. With the addition 
of the SoniWing, the user’s entire body moves within 
the virtual environment, allowing greater freedom than 
systems restricted to head motion, and the environment 
responds realistically to the user’s location and move- 
ments. Sensics also offers customized systems that track 
a user’s eye movements, which help vision researchers 
learn which images or objects in a scene hold a user’s 
interest, and also offers a video see-through solution for 



An operator in the background wears a head-mounted display that allows him to see through the Robonaut’s eyes 
(cameras) with a wide near-human field-of-view. 
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The lightweight piSight head-mounted display offers a wide 
field-of-view with multiple screens per eye to allow for more 
realistic visual experiences. 

applications in which users need to see the virtual world 
and reality simultaneously. 

The piSight HMD is light (weighing less than 
2 pounds), stable, comfortable, and adjustable to an indi- 
vidual’s head and eye shape, but Sensics still wants to make 
the technology more lightweight and efficient. Currently, 
piSight can be used with a variety of virtual reality soft- 
ware and platforms, including OpenSceneGraph on 
Linux and WorldViz products on Windows. 

The piSight also offers high resolution “better than an 
HDTV display,” according to Brown, with the NASA 
model displaying 12 screens per eye; lower-end models 
have as few as 3 screens per eye. Other commercially 
available HMDs usually offer fewer screens, fewer 
cameras, or poorer resolution, which leads to either a 
much smaller field of vision (i.e., no peripheral vision) or 
a poor-quality image that appears pixilated or stretched. 
With more screens per eye, however, each display can 


offer a slightly different orientation, forming a concave 
arrangement that wraps around the eye, and projecting 
a different and more realistic perspective. In a higher- 
end model, the user can see in high resolution for 130 
degrees, which is almost as wide as the natural range for 
humans. “The operator basically sees a field-of-view that 
has virtually no limitations,” Brown notes. 

The piSight camera array captures panoramic, high- 
resolution live video, which a user views while wearing the 
piSight HMD. Video is compressed, sent over a network, 
and then displayed inside the HMD. This interface may 
be able to support astronauts and engineers on Earth with 
various tasks like repairs and maintenance in high-risk 
environments. An astronaut in orbit, for example, could 
use the piSight remotely to control a robot on the surface 
of the Moon and examine the surrounding environment 
in real time. The system could also enhance safety on 
Earth by enabling remote operation of machinery in 
biohazard, defense, or medical environments, without 
requiring physical access to the site. In an emergency 
involving hazardous materials, a human operator could 
use the piSight system to remedy the situation from a 
safe location. 

Sensics is finding success in the commercial sector, with 
sales of its products topping $1 million in 2007. Sensics 
is currently marketing the piSight to larger companies, 
since system prices start at about $27,500. Depending 
on customization and features, such as additional camera 
arrays for a wider field-of-vision, the units can cost up 
to $100,000. 

Commercial applications for the piSight are only 
limited by customers’ imaginations. “Customers have been 
selecting the piSight systems for a variety of simulation, 
training, design, and research applications,” says Brown. 
An automobile designer uses the HMD to “sit” in the 
driver’s seat of an automobile and explore the design 
in a realistic, immersive experience before production 
begins. Other companies are beginning to use the piSight 
for training purposes that involve learning specific 



The piSight can be used with Linux or Windows virtual reality 
software, and can be controlled via motion trackers 
or joysticks. 

tasks in situations, such as teaching a quarterback a 
specific play or showing a soldier how to operate a certain 
piece of equipment. Clearly, NASA will not be alone 
in reaping the benefits from virtual reality systems like 
the piSight. ♦♦♦ 

piSight™ is a trademark of Sensics Inc. 

SoniWing™ is a trademark of InterSense Inc. 

Linux® is a registered trademark of Linus Torvalds. 

Windows® is a registered trademark of Microsoft Corporation. 
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